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ABSTRACT 

\ This study evaluates the assessment center approach 
applied in an educational setting to the selection of secondary and 
elementary school administrators. The performance criteria used 
included performance ratings by three different groups (supervisors, 
teachers, and support staff) a*nd school cl imate ratings from three 
different groups; (teachers, students, and support staff). The results 
show a significant relationship between the assessment center 
placement recommendations and the supervisor, teacher, and support 
staff ratings. The school climate ratings (by group) did not 
significantly correlate with the assessment center ratings. The 
author notes that the data in this study represent the first 
validation of the use of assessment center technology in the 
selection of school\ administrators and suggests, that assessment 
centers should be useful selection devices in most educational 
settings. The report includes a list of references and six tables. 
(MD) 
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Abstract 

The research described in this paper was directed to ^he validation of an 
assessment center used to select school administrators. Criteria included 
ratings on behaviorally anchored rating scales by supervisors, teachers, and 
support staff and measures of seven school climate dimensions provided ;by 
teachers, students, and support staff. Results indicated a significant. ' 
relationship between an overall assessment center placement recommendation 
and supervisory, teacher, and support staff ratings on most dimensions. Climate 
measures were significantly related to few assessment center ratings regardless 
of the source of the climate measure. Of primary importance is the fact that 
assessment center ratings proved valid in a new setting, though . the use of . 
performance ratings as opposed to promotion or salary increase criteria has 
implications for the Klimoski and Strickland criticism of assessment center 
validation research. 
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Assessment centers are extraordinarily popular among professionals and 
practitioners involved in the selection of managerial personnel; This popui- 
larity is reflected in a large number of published papers (e.g., Finkle, 1976; 
Huck, 1976; Klimoski & Strickland, 1977), the publication of standards and 
ethical considerations involved in assessment center operations (Task Force on 
Development of Assessment Center Standards, 1977), and the unusual support of 
the Federal courts (Byham, 1979)/ This enthusiasm has been generated to a 
great extent by the positive research findings concerning" the validity of the 
assessment center. Huck (1976) reviewed over 50 studies all of which reported 
positive findings concerning the relationship between assessment center ratings 
and subsequent job performance. Klimoski and Strickland (1977) reviewed over 
90 studies finding that validities vr.re positive and high (median = ,40) with 
advancement criteria (number of promotions) , and that very few additional studie 
had been done after 1972 indicating general acceptance of *the validity of assess 
ment center methodology. 

Studies concerning the internal validity of the assessment center (Huck, 
1976; Mitchel, 1975; Schmitt, 1977; Schraitt & Hill, 1977) indicate that inter- 
rater reliability is high and that overall assessment judgments are indeed 
highly correlated with performance on individual assessment dimensions 
(Sackett 6rHakel, 1979; Schmitt, 1977). Sackett (1982) has raised a number 
of issues concerning the degree of understanding we have of the rating process 
and has indicated that a mechanical combination of ratings to produce an over- 
all rating night be mere practical than the consensus discussions usually 
employed (Sackett & Wilson, 1982). However, these possible problems relate 
more to the mechanics of operation of a center than they do to the psycho- 
metric, quality and external validity of assessment center ratings,. 

Validity studies have generally been sound in that they are longitudinal 
and that assessment center ratings have been withheld from organizational 



, - Assessment Center Validity. 

'. 3 
personnel who mlfclt later Influence the candidates 1 promotions, thus diminishing, 
-if not eliminating, the possibility of criterion contamination. The research 
presented in this paper represents, an evaluation of the assessment center 
approach applied In an educational setting to select secondary and elementary 
school administrators. While there have been previous educational applications 
of the assessment center, there are no previously published empirical repdrts 
of the validity of the technique in this context. Most previous studies of 
assessment center validity have employed promotions or salary progress as 
criteria (Klimoski & Strickland, 1977). In this study, the criteria of per- 
formance included ratings by three different groups of people as well as school 
climate ratings collected from three different groups. The research, then, 
represents an extension of currently available information in two respects: 
educational administrators and extension to new performance criteria. 

Method 

Sample . The sample included 153 school administrators who had participated 
in an assessment center between the spring of 1976 and the spring of 1981. These 
people were candidates for promotion to assistant principal or principal in 
elementary, junior high, middle, or high schools in seven different school 
districts throughout the country. Sixty of these individuals were subsequently 
promoted to either assistant principal or principal. The remaining 93 already 
held assistant pr incipalships and were included for several reasons. First, 
they were performing as administrators and therefore criterion data could be 
collected. Second, they were actual candidates for promotion hence their 
motivational set in center participation would be similar to any other candidate. 
Third, the assessment cneter ratings were used to select people in all seven 
districts; these 93 individuals represented a group who were not necessarily 
\the best performing people; consequently, restriction of rangeeffects result- 
ing from the use of ratings to select should be less important. General 
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demographic description of the subjects is presented in Table 1. As a minimal 
qualification, all candidates for administrative positions had completed some 
graduate vork in educational administration; most had masters degrees. While 
total sample size was 153, sample size for many reported correlations was 
substantially less because rater respondents felt unable, or were unwilling, 
to give ratings on a particular dimension. 

Insert Table 1 about here „ 
Assessment Center Dimensions . and Use . The assessment center was developed 
for the National Association of Secondary School Principals (NASSP) with the 
voluntary help of the APA Division of Industrial and Organizational Psychology . 
(see Jeswald, 1977; Moses, 1977). Since that time NASSP has provided - a -staff 
consultant to provide advice and assessor training to interested school districts. 
A job analysis conducted prior to the development of the center indicated that 
twelve dimensions of behavior were important for successfully working school 
administrators. It was felt that these dimensions could be assessed in an 
assessment center. These dimensions and their descriptions are listed in 
Table 2. Three of these dimensions » (Personal Motivation, Range of Interests, 



and Educational Values) were included for personal self development not as 
skill measures. 

Insert Table 2 about here / 

In addition, a summary Placement Recommendation was also made. -This was an 

overall rating mace at the exclusion of the assessors' integration session 

and represented the assessor^' overall appraisal of the candidates' potential 

as school administrators. Assessing these dimensions involved the use of two 

inbaskets, a semi-structured interview, a fact-finding and decision-making 

simulation with an oral presentation, and an analysis and group discussion of 

a case study. All ratings were made on Likert-type five point scales. Consensus 
ratings of the skill dimensions as well as the Placement Recommendation were 
completed after a two day discussion of candidates by the assessor team. These^ 
q consensus ratings were used as predictors of job performance and school climate ratings 

ERJC g 
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^sisessment center ratings were used by .the personnel 'departments in all . 
districts to make recommendations concerning promotion. \ In no instance, how- • 
ever, were actual assessment ratings provided to any individual who subsequently 
provided performance data. A total of 425 persons were assessed during this 
period in all, the: dis.txlc.ts. of ..whom -on-ly-60-were— promoted; — Sinbe assessment 
center ratings were used in making these promotions there was severe range 
restriction. Fortunately, 93 additional assessment center candidate^ held 
administrative positions (assistant principals)' and these persons were also 
included in the validation sample as indicated above. The remaining 272 
candidates did not hold administrative jobs at the time performance data were 
collected, hence could not be included in the validation study. 

Criterion f-feasures . Job analysis interviews were conducted in thirteen 
school districts throughout the country. These interviews included principals, 
students, parents, teachers, support staff personnel, and superintendents. The 
pertinent outcome of these job analysis interviews for this study was the develop- 
ment of fifteen behaviorally anchored rating scales. - These scales (see Tabl^e 3 
for brief titJLes) plus an overall rating were used as one set of criterion 



measures. These measures were administered to the supervisor of each of the 
153 administrators as well as two teachers and two support staff personnel in 
the administrator's school. Raters were directed to decline to make a rating 
on any scale about which they felt they did not have sufficient information. 
This option was frequently used by the support staff personnel because of their 
lack of opportunity to observe the principal's performance on various dimensions. 

During the job analysis interviews, various groups were asked about 
critical aspects of the principal's job. Many responded that the principal's 
primary responsibility was the establishment of a "good" school climate. In 
operationalizing this construct, we developed seven climate measures for 
different job performance dimensions. Specifically, two teachers, two support 
staff, and four students in each school were asked to describe the school 

ERIC 7 
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climate relating to Curriculum issues, Student Activities, Support Services, 
Staff Selection, Evaluation and Development, Community Relations, School Plant 
Maintenance., and Structured Communication. Examples of items in these scales 
which referred to conditions at the school include "I feel informed about the 
school organization , operation, and rules" and "It is easy, to get by with 
cheating at this school." Since -they referred to conditions at the' school as a 
whole, the climate measured were not simply alternate measures of performance 
ratings. Responses to the 55 items were made on a scale with five options: 
1 p Strongly Agree, 2 = Agree:, 3 - Disagree, 4 = Strongly disagree, and 
5 = No opinion/no experience.. Student leaders, teachers and support staff 
completed the climate questionnaire, and teachers, support staff, and the . 
administrator's supervisor completed the performance ratings. District " ' 

personnel administrators chose the support staff* teacher, and student raters. 
The researchers instructed them to choose "experienced and informed" individuals 
who would be familiar with the principal's work and who carried' out their daily 

activity within the participant principal's school building.' Finally, it is 

A 

important to note that different groups of people supplied performance "and- 



climate ratings thus avoiding the possibility that similar response sets 
contaminated both measures f . 

All responses to performance ratings and climate questionnaires were re- 
turned to the personnel administrator in charge of the project in each district, 
and then mailed, unopened, to the research group at Michigan State University. 

Results 

Performance Ratin gs. As stated above, performance ratings were made by 
three separate groups: Supervisor , teachers, ~and support staff. While the two 
teacher ratings and the two support staff ratings were intercorrelated at a 
level supporting their combination, it vas-felt that correlations across the 
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three rater groups indicated they were 0 perceiving performance in a significantly 
different way. The intercorrelat ions of the performance ratings from different . 
rater groups as well as the validities of these performance ratings against the 
Placement Recommendation are presented in Table 3. Since the Placement Recommenda- 
tion was used to make promotion decisions only correlations, with the Placement 
Recommendation are presented, Correlations of performance ratings with all 
assessment center ratings as well as the intercorrelations of the ratings are 
available from the senior author. The Placement Recommendation was significantly 

Insert Table 3 about here 
related to all but two (Supervision of Student Activities and Interpersonal * 
Effectiveness) of the performance dimensions as rated by supervisors. The 
Placement Recommendation was also correlated to teacher ratings of performance 
with three exceptions. Two of these (Fiscal Management and Maintenance of 
School Plant) may be dimensions which are more important to supervisors. Support, 
staff ratings of performance were not as highly related to Placement Recommenda-^ 

ti ons but significant rel ationships were observed for se ven of the _ _sixt_een- 

performance ratings. Most correlations with all three sets of ratings were in 
the .20 to .35 range. 

Those assessment center dimensions most highly and consistent ly. correlated 
with job performance were Leadership, Oral Communication, Organizational Ability, 
Decisiveness, Judgment and Problem Analysis. Correlations of ratings on 
Sensitivity, Stress Tolerance, Educational Values, Range of Interests, and 
Personal Motivation with various job performance . ratings are lower and, for 
the most part, nonsignificant. * • • 

Climate Ratings . All seven of the climate measures were assessed by 
multiple items and multiple raters; hence, both interitem and interrater con- 
sistency were evaluated. Interitem consistencies were in the' .70 f s and .80 f s, 
but interrater correlations were very low (frequently less than .30) especially 



: _ * A«sfi»-yi<ir,pnt t'ent or V.il I<llly 

.8 

across rater groups. Consequently, student, teacher, and support staff 

climate ratings were computed for each of the seven scales and these were 

correlated with the assessment center ratings. Interdimension'al correlations 

were high relative to the interitera consistencies suggesting a large general . 

climate factor, though each conceptual dimension was identified by some 

specific variance. Tables containing internal consistencies, interrater 

reliabilities and scale intercorrelations are contained in-Schraitt and Noe 

(1982) and are also available upon request from the senior author. 0. The corre- 

1 a t i on s . _be twe e n t h e a s s e s s me n t cent e r . consensus ski 11- r a t in g s a n'd _ s t u d e n t , 

teacher, and support staff perceptions of the* school climate are presented in 

Tables '4-6. " Correlations are all generally low and nonsignificant regardless 

Insert Tables 4-6 about here - * 
— — — — — _ " — _ _\.'__ _ _ _ _ 

of the source of climate information and skill dimension considered. 

Post hoc consideration and interpretation o,f significant correlations 
given the large number computed is certainly questionable, but the following 
stat ements "a re off ered as possible testable hypo theses in a ny_JEutu r <* _e-f_f-n rt 
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to relate skill and climate variables. Range of Interests was significantly 
correlated with student perceptions of School Plant Maintenance , Curriculum 
and the extent to which the principal Structures Communications. It may be 
that a person with a large range of experiences and interests can better deal 
with the variety of demands faced by an administrator. Further, -it may be that 
the memos, announcements, and bulletin boards used by the administrator, to 
communicate reflect the diversity of her/his interest and ability. For' teacher 
responses,- a climate stressing student activities was significantly related to 
three assessment center ratings — Educational Values, Personal Motivation, and 
Range of . Interests . Again, a post hoc interpretation might be that teachers 
are Sensitive to the breadth of an administrator 1 s interest, values, and 

motivation as they are reflected in the programs and involvement in student 

\ 

activities. ' '{ 0 
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■ Table 6 illustrates the correlations between the consensus ratings and 
^upport staff ratings of school climate. WrLtten communications is related 
to support staff climate ratings of Student Activities , Support. Ser vice s , a_nd 



Staff, Selection Evaluation and Development, Support staff personnel are . , ; 
greatly involved in administrative work (typing memos, filing, running errands) 
which ^is delegated to them by the principal. Their jobs involve direct contact 
with the principal's written work. Therefore, their perceptions of the prin- 
cipal's involvement in student activities, directing support services, and 
staff selection, evaluation and development are likely to be directly related 
to the ability of the principal to express ideas clearly in writing, in a 
manner appropriate for. students, administrators, parents, and teachers. How- 
ever, our. original hypothesis that an administrator's skill would be related 
to the establishment of a climate for specific types of behaviors in a school 
(i.e., behavior related to curriculum considerat ion , school maintenance , 
student activities, etc.) is certainly not strongly supported by the data in 
Tables 4-6. 



Discussion and Conclusions 

There are several reasons why the data presented in this study are impor- 
tant and interesting. They* represent , to our knowledge, the first validation 
of the use of assessment center technology to the* selection of school admin- 
istrators. The positive validation results using a sample from widely distant 
geographic areas and scho.ols/districts of diff<- "iug' sizes and levels as well 
as the job analysis 'results indicate little di: r erence in job responsibilities 
and tasks across various districts. It suggests that assessment- centers 
should be useful selection devices in most, if not all, educational settings. 

The items comprising the climate questionnaire did not call for evalua- 
tions, of the administrator, but rather evaluation of the school. They, indicate 
that the skills of the principal (as measured by the assessment center) are 

9^. 11 
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generally not related to overall school functioning as perceived by students, 
teachers, o% support staff . This may be duetto the fact that the principal 
alone is not responsible for the school's functionin g . The skil ls o f dist rict^ 
level personnel in receiving monetary assistance from state government- and the 
attitude of community members toward the school and their involvement in its 
functioning may be factors which have greater influence on the** school climate 
than the skill level of the principal. These situational factors may be 
directly responsible for the lack of validity between the climate ratings and 
the principal's skills as measured_Jby the assessment center. 

The validity of the assessment center ratings extends across more dimen- 
sions of the principal's job performance for teacher and supervisory ratings 
than for support staff ratings. Post hoc , the explanation may be that some 
of the performance dimensions • involved aspects of the administrator's job 
which were indirectly observed or for which the support staff had only informa- 
tion from other people. Many of the support staff chose not to make ratings on 
. s ev eral dimensions , but even those who did may not have bee n in f o r med_jra t ers . 



The data presented in this paper are also relevant to the criticism ttiat^ 
assessment center ratings are simply expensive ways to capture an organization' 
promotion policy (Kliraoski & Strickland, 1977) since most validation studies 
have employed promotion or salary increase criteria. Significant validities 
were observerd for most performance dimensions provided uy supervisors, teachers 
and support staff. This suggests - that assessment centers are valid predictors 
of job performance not merely indicators of an individual's ability to survive 
within an organization. Because ratings of Performance are also judgmental, 

« s 

they may be susceptible to the same criticism Klimoski and Strickland raised 
concerning these more traditional criteria in assessment center validation 
research. However, the' subordinate ratings collected in this study would seen 
less susceptible to this problem., 
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The study has some limitations, which probably- did serve to decrease the 
size of the observed validities. First, average job tenure of the promoted 
participants was only thirteen months. While tenure for those who' were already 
administrators at the time of assessment was much longer, there was a sub- 
stantial portion of the group who had been administrators for a relatively ■ 
brief period of time. Performance for administrators probably does not level 
off until much later (Schmitt & Schneider, 1983), and the raters (particularly 
supervisory raters in this study) may not have had a great deal of opportunity 
to observe the participants' behavior in a year's time. Restriction of range 
resulting from the use of the center ratings was not a significant problem 
because of the inclusion of individuals vho had been lower , level administrators 
but who had-, not been promoted as a result of the center. The standard deviations 
of the Placement Recommendation for selected and unselected groups were .77 and 
.85 respectively. Use of these standard deviations in the correction for 
restriction of range yielded corrected validity coefficients about .03 higher 
than the observed coefficients. Perhaps the most severe limitation of the study 
lies in the unreliability of the climate ratings alluded to above, especially 
for support staff and teachers. 

Mote work should be done using performance criteria such as turnover and 
absences, grievances filed, and achievement of management "objectives in the 
validation of" assessment centers. While we do not advocate a naive search for 
some ultimate criterion, the use of these criteria would 'inr-ease current 

knowledge concerning the construct validity of the assessment center ratings, 

' ... * ' 

particularly in determining behavior-criteria relationships. Because 6f the 
overreliance on promotion and salary growth as the sole criterion in establishing 
assessment center validity, much potentially useful information concerning . 
assessment center behavior and job behavior has not been forthcoming. 
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Table 1 



Demographic Characteristics of Validity Study Participants 



Sex. 

91 
62 



129 
24 

School at time of assessment 

Elementary 65 

Jr. High/middle 24 

Senior High 60 

District Staff 4 

Position at time of assessment 

Teacher , 

Counselor/Ed. Specialist 
Assistant Principal 
Unknown \ 

' \ 

Number of participants who\have been promoted 

\ 

\ ■ 




Male 
Female 



Race 



White 
Other 



A3 
25 
81 
A 

60 
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Table 2 

Assessment Canter Dimensions and Descriptions 

1. Problem Analysis . Ability to seek out relevant data and analyze 
complex information to determine the important elements of a problem 
situation; searching for information with a purpose. 

2. Judgment . Skill in identifying educational needs and setting 
priorities; ability to reach logical conclusions and make high- 

quality decisions based on available information; ability, to 

critically evaluate written communications. 

3. Organizational Ability . Ability to plan, schedule, and control the 
work of others; skill in using resources in an op timal fashion ; ability 
to deal with^a volume of paper work and heavy demands on one's time. 

4. Decisiveness . Ability to recognize when a decision is required and to 
act quickly. (Without an assessment of the quality of the decision.) 

5. . Leadership . Ability to recognize when a group requires direction, to 

get others involved in solving problems, to effectively interact with 
a group, to guide them to"~the accomplishment of -a task» „ 

6. Sensitivity . Ability to perceive the needs, concerns, and personal 
problems of others; tact in dealing with persons from different back- 
grounds; skill in resolving conflicts; ability to deal effectively 
with people concerning emotio nal issues; knowing what information 

to communicate and to whom. 

7. " Range of Interests . Competence to discuss a variety of subjects 

(educational, political, economic, etc.); desire to actively partici— 
pate in events. 

8. Personal Motivation . Showing that work is important to personal 
satisfaction; a need to achieve in all activities attempted; ability 
to be self-policing. 

9. Educational Values . Possession of well-reasoned education; 
philosophy; recept iveness to change and new ideas. 

10- Stress Tolerance . Ability to perform under pressure and opposition; 
ability to think on one's feet. 

11. Oral Communication Skill . Ability to make a clear oral presentation' 
of ideas and facts. 

12. Written Communication Skill . Ability to express ideas clearly in 
writing; to write appropriately for different audiences—students, 
teachers, parents, other administrators. 
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Interrater Reliability of Performance Ratings 
and Validity of Placement Recommendation 
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Correlation Between Assessment Center Consensus Ratings 
and -Teacher Ratings of School Climate 
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Cor mint ion Between Asseefii-ent Center Consensus Ratings 
and Support Staff Ratings of School Climate 
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